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1 . To  learn  oynt.-ictlc  oatterr,.': . 

The  follov/inr,  etudy  was  started  followinc  discussion 
with  Leon  Cooper,  V.'altcr  Frelbcrcer,  and  Henry  Kucera.  The 
topic  had  been  lanr.uaco  learning,  how  children  learn  a languace , 
and  to  v/hat  dcf-ree  it  is  possible  to  construct  a model  - an 
abduction  r.achine  - that  can  perform  such  tasks. 

Wo  sinr.lod  cut  one  part  of  language  learning,  namely  to 
discover  the  syntax.  This  means  that  we  leave  aside  the 
intricate  problems  preceding  syntax  learning,  such  as  speech 
segmentation,  learning  and  recognizing  words,  and  semantic 
association . 

Also  v;g  vs'ould  of  course  try  any  abduction  .machine  on 
formal  languages  rather  than  natural  language.  It  seems 
.natural  to  start  v.'ith  some  .simple  classes  of  languages. 

A question  that  arises  immediately  is  v/hethcr  our  problem 
is  solvable  in  the  sense  of  logic.  It  is  well  known  that  so.me 
seemingly  simple  questions  in  formal  languages  are  not  solvable. 
To  mention  just  one  example:  it  is  unsolvablc  If  the  languages 
produced,  by  tvio  context  free  grammars  are  Identical. 

How'evor,  this  is  not  what  we  are  really  trying  to  find  out. 
Indeed,  if  we  could  construct  an  abduction  maciuno  that  per- 


formed its  task  in  principle  but  required  an  enormous  co.mputing, 

( 

effort  to  do  it,  then  vfe  would  not  consider  this  a satisfactory 


solution.  Also  v/e  would  require  that  the  functioning  of  the  ■ 


machine  should  bo  natural  in  the  sense  that  its  ooeration 


resemble  the  way  that  humans  learn  syntax,  or  at  least  does  ■ »»«/■  vwu. 


not  see.m  completely  artificial. 


'I 
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On  the  other  hand,  if  we  had  an  unsolvablc  problem  at 
hand,  but  could  produce  a machine  that  produced  a Icarninc  erfect 
with  reasonable  coinputlne  effort  and  with,  results  that  approx L- 
nate  the  true  rp.-amnar,  then  wc  may  bo  wlllin;:;  '>^0  accept  it  as 
a solution  to  our  problem. 

In  other  v/ords,  what  v,'e  are  after  is  not  competence  but 
performance . Come  important  consequence  of  this  decision  v;ill 


turn  up  in  lator  sections. 

It  is  obvious  that  the  amount  of  preproi3ramming  of  the 
neural  net  that  is  required  for  this  task  will. depend  upon  how 
general  is  the  class  of  gramir.ars  that  v;e  allov;.  ?or  the  present 
paper  v;e  shall  assume  that  the  grammars  are  at  least  context  free 
and  most  of  the  tiir.o  even  finite  state  grammars. 

The  neural  machines  used  v;ill  resemble  the  ones  we  have 
used  in  earlier  v;ork.  They  can  still  be  considered  as  dynamical 
sytems  in  a v;ide  sense,  but  v/here  the  system  is  not  represented 
by  ordinary  differential  equations  but  have  the  structure  of 
automata. 

We  shall  perform  experiments  that  will  help  us  understand 
how  abduction  machines  should  be  constructed.  I’cr  this  purpose 
it  v;ill  be  convenient  to  have  available  a program,  that  will  help 
us  set  up  the  graimmar  easily  and  will  generate  sentences  accord- 
ing to  it. 

The  model  used  will  be  the  one  introduced  in  Grenander  (1966) 
and  studied  in  some  later  reports,  k’ith  a given  lexico.n  of 
words  a,b,c,...  wc  postulate  context  free  rules  of  the  form 


(1.1) 


V - a trine 


r 


whore  V ic.  cii'^  Oi  the  syntactic  variables  and  strine  is  made 
up  of  •.-.•ords  and  vorLables.  The  variables  will  be  labeled  by 
nu’nboi's  I,.',;;,...  . Tor  a Given  variable  v we  assume  a 
probability  d Ls': Ibution  over  the  rules  that  rov/rite  that 
syrribol.  ’.•.'o  hav*^  s’nown  elsewhere  what  properties  those 
probability  distributions  must  have  in  order  that  the  probability 
measure  induced  over  the  set  of  all  finite  v/ord-strinGS  be  a 
real  one  in  the  sense  that  this  set  have  total  probability  one. 

It  will  be  assumed  that  these  properties  hold  for  the  G”2.mmars 
we  define.  V/e  then  speak  of  a syntax- con  trolled  orobabilit  y 
Grammar . 

A convenient  vehicle  for  doinG  this  is  the  A?L  program 
SETUP  given  in  the  Appe.ndix.  It  interrogates  the  user  about 
the  size  of  the  lexicon,  what  syntactic  rules,  v/hat  probabilities 
associated  with  the  rules  and  so  on.  Then  the  function  -REWRITE 
will  generate  one  gi-aiamatical  sentence  accerding  to  the  model. 
Repeated  calls  of  REWRITE  will  give  us  a sample  from  the 
language . 

It  will  be  instructive  for  the  user  to  generate  a sample 
and  see  how  a human  would  try  to  discover  tlio  underlying  grammar. 
It  is  not  so  easy,  even  for  simple  grammars.  Wc  now  turn  to  the 
main  question:  construe  a machine  that  docs  it. 


ii 


2.  A n .?■  I V e ab cl u c t low  machine. 

Hov;  does  soncone  learn  a lancuace  v;lthout  explicit  irsstruc- 
tion  in  its  children  c^oup  words  into  word  classes, 

employ  conceptr.  like  parts  of  speech,  search  for  rules? 

Whether  this  is  os  or  not  it  is  at  least  true  that  this  is  the 
way . Grammarians  have  been  operating  from  Panini  and  Thrax  on. 

Let  us  try  to  imitate  this.  To  put  it  more  for.mally  let  us 
consider  tv/o  strings  u and  v of  v.'ords . The  strings  need  not 
themselves  be  grammatical  sentences.  If  it  is  true  for  any 
pair  x.y  of  strings  that  xuy  and  xvy  arc  simultaneously 
grammatical  or  ungramm.atical  the  original  strings  are  said  to 
be  equivalent,  written  as  uEQv.  Obviously  TO.  is  an  equivalence 
relation  and  it  divides  the  set  of  all  finite  strings  into 
equivalence  classes.  This  is  an  established  approach  in  formal 
linguistics. 

To  simplify  the  discussion  we  shall  assume  that  our  language 


i/  is  a finite  .state  language,  which  is 
that  it  is  generated  by  a finite  state 
Then,  after  some  simple  changes  all  the 
form,  cither 


the  same  thing  as  to  say 
grammar  G , 1/ ' = L ( G ) . 
syntactic  rule  take  the 


C2.1)  i - aj 


where  1 and  J arc  syntactic  variables  and  a is  some  word  from 
the  lexicon,  or 


C2.2) 


a . 


0 


the  machine  Ic  in  the  final  state  F.  The  function  SETUP 
Renerated  a sample  bccinninc  with- 


(2.4) 


A ADC 


AABDAAABBAADC 

BDABBABBD 

DBABC 


ABBA/iP^C 


AAAAAA3C 


For  a rif'ht  linear  need  only  consider  uy  and 

vy  to  define  equivalence.  If  u = ab  and  v = b it  is  seen  that 
•with  these  two  strings  as  a beginning  the  automaton  in  Figure 
2.1  v/ill  bo  in  state  2.  V.'hatever  follou's  it  is  clear  that  the 

result  -will  , bo  either  grammatical  or  ungra;nrr.atical  so  that 

aEQab . They  belong  to  the  same  equivalence  class.  On  t.he  other 
hand  the  string  bb  ends  in.stabe  3 so  that  bb  belongs  to  another 
equivalence  class.  The  string  cb  cannot  be  generated  by  the 

machine  but  it  will  bo  convenient  to  add  a nev;  state  to  take 

case  of  such  strings  and  their  equivalence  class.  Then  it  is 
seen  that  the  states  of  the  automaton  and  the  associated  syntactic 


variables  correspond  directly  to  the  equivalence  classes. 
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Ccnnldcr  no’.v  tho  reacticno  of  the  Imagined  child  to  itc 
lansuaco  environment.  V/hen  it  listcr.o  to  preoumabiy  c^^’-Ti-’Tiatical 
Gontcnccr.  it  would  look  for  equivalence  cla.cooo . To  repreoent 
them  it  Ig  cnouf.ii  to  select  one  string  frcr.i  each  arid  short 
strings  will  be  adequate  unless  the  nuiriber  of  variables  is  quite 
large.  Say  that  one  considers  the  set  of  all  strings  of  length 
at  most  equal  to  d,  for  depth,  as  candidates. 


The  maximur;;  number  of  equivalence  classes  that  could  be 


found  in  this  v/ay  is 

(2.5)  . n^,  + + n3  + 


Note  that  n increases  fast  with  d. 
c 

As  sentences  are  being  encountered  the  listerncr  cries 
the  initial  substrings  up  to  depth  d as  candidates  for  the 
distinct  equivalence  classes.  A sentence  ux  is  compared  with 
earlier  encountered  sentences  of  the  for.m  vx . If  they  can  be 
found  in  .memoi’y  it  also  looks  for  sentences  uy  and  vy.  If  an 
Instance  is  found  in  v.'hich  uy  is  grammatical  but  vy  not  it  is 
clear  that  u and  v G.re  not  equivalent. 

Create  a matrix  n xn  and  put  a large  ncgacivo  number  in 
tho  cell  corresponding  to  the  indices  of  u and  v each  time  the 
above  event  occui-rcd;  this  signifies  that  u and  v belong  to 
different  classes.  On  the  other  hand  if  uy  and  vy  arc 
grammatical  for  some  y increase  tho  value  ih  tho  cell  one  unit. 


The  latter  iz  done  by  ooc-aklnr.  The  nentence  vy  is 
spoken  and  if  it  Is  accepted  by  the  environnont  as  correct  the 
cell  value  of  the  matrix  is  increased  as  mentioned . 

As  time  r.ocs  on  the  partition  vfill  bo  finer,  lack  of 
equivalence  v;ill  be  established  v/ith  certainty,  v/hlle  equivalence 
only  is  gradually  increased.  V*e  nov;  apply  a threshold  logic. 

If  u and  V have  a cell  value  which  is  positive  we  treat  them 
(temporarily)  as  equivalent.  This  will  net  necessarily  be  a 
true  equivalence  relation  since  it  need  not  be  transitive.  V.'e 
therefore  have  to  extend  it  by  forming  chains  by  pairwise 
equivalent  pairs  of  initial  strings. 


Ine  matrix  is  initialized  by  pu* 


all  entries  ecual 


a moderately  large  negative  number:  the  tab'.iia  rose  hypothesis. 
The  flov;  chart  looks  as  in  Figure  2.2. 

The  block  SPEAK  AKD  TEST  involve  a good  deal  of  computing 
with  comparisons  and  matching.  Here  we  need  a "teaching" 


program  called  ACCEPT,  see  Appendix,  that  will  also  bo  used  in 
a more  ambitious  abduction  m.achine . 


This  machine  was  coded  and  run  with  the  fo 
enecs.  The  machine  worked  in  the  sense  that  for 
num.ber  of  variables,  they  and  the  corresponding 
eventually  discovered.  In  spite  of  this  it  was 
failure  for  thi'oe  reasons. 

1.  Even  in  these  extre.mely  simple  cases  1 
exasperatingly  slow. 


llowing  experi- 
vei'y  small 
rules  were 
deemed  a 
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2.  The  space  requirement  v.'as  grovflnq -at  an  unacceptable 
rate  as  each  (nev;)  sentence  heard  had  to  be  stored  in  its 


memcry 


Iris  is  very  unlikely  in  human  learning, , perhap: 


impossible  because  tlie  storacc  volume  required. 

3.  The  search  in  SPEAK  AKD  TEST  and  the  rollov.’inc  blocks 
in  Fisure  2.2  is  too  systematic.  V'hatevcr  v.'ay  humans  .may  use 
to  learn  (grammar  this  is  not  it . 

We  therefore  decided  to  scrap  this  abduction  .m.achine  and 
build  a better  one. 


3 . A smarter  machine . 

The  idea  to  look  for  equivalence . classes 
but  not  the  v/ay  it  v.'as  done.  To  visualise  th 
needed  for  the  computins  in  Figure  2.2  look  a 


3 u .111  scorr.s  or‘crr..l 
e neural  .netv.'ork 


t.ne  a: 


..  ci... 


s 


Fi.qure  3 • 


Hero  n, , = 3,  cl  = 2,  n “ 12.  V’e  set  up  a connectivity  matrix 
of  size  12-'<12.  V/o  do  not  need  more  than  the  entries  above  the 
main  diarronal  '.s'hich  moans  n^(n^-l)  values,  in  the  picture  - 132. 
At  TAnUTiA  I\A.''A  all  tliesc  values  v/ill  be  moderately  larc;c 
necativc  numbei-s  v.’hich  will  be  updated  following  the  results  of 
SPEAK  AND  TEST.  Already  for  slightly  larger  values  for  n,_,  and 
n.^  this  would  lead  to  enormous  storage  reCiUirement . 


’.'.’hat  is  v/rong  in  this  machine  is  th 


eroceoa:'.  bv 


aggregation : starting  from  lots  of  possible  equivalence  classes 

they  are  coalesced  into  fewer  until  we  have  arrived  at  the  true 
number  of  classes. 


I.nstead  wc  should  start  from  the  oooosite  end. 


Assuming 


that  there  is  only  one  equivalence  class  Cti.o  function  T.A3ULA 
in  the  Appendix)  the  machine  listens  to  grammatical  sente.nces. 

It  takes  initial  substrings  and  determines  its  num.bar  and  class 
(from  t.he  updated  matrix  CLASS).  This  class  may  contain  several 
othei  substrings.  One  of  them  is  selected  at  rando;ri  (no 
systematic  search)  and  it  will  be  tested  for  equivalence  with 
the  first  one.  Depending  upon  the  outcome  of  the  comparison 
either  a new  variable  (row  in  CLASS)  is  created,  or  a substring 
is  moved  from  one  row  to  another,  or  no  action  Iz  taken.  All  of 
this  is  done  by  the  program  LI.N’GUA  which  calls  the  comparison 
function  Tl'.ST,  sec  Appendix. 

V,Q  now  claim  that  this  abduction  machine'  is  consistent  in 
the  following  ecrfcrmancc 


sense . 


Theorcn.  CoriGiuor  tho 


ho  f lov.-charf;  i.n  i'lr.urc 


describod 


Aanuir.o  that  D^PTd  d blio  dcnth  of 


rue  r 

■rn!’ii:;a 

!•.  As 

he  total 

number  ol 

i te  ra t ons  , 

TOTAL , 

t 

o .! 

. n !'  i.  ri  i 

tv  lea 

rn 

ing  of  t 

i'iC  grammar  occi.;:’.-,  w!t;'. 

nrobablli 

e:  the  number  of  classen  and  the  clansoa  thornrcl vor.  will 


convcrjic  to  a limit  ouch  that  it  and  the  cc 


form  a rna,-ur.ar  "•■"•ui valent  v/lth  the  true  c . 

Proof : Since  the  ocarch  is  not  systematic  but  (to  some  extent) 

random  any  convergence  must  be  of  a probabilistic  kind. 

Say  nov;  that  for  a large  number  of  T a certain  set  of 
classes  have  been  established.  As  nev;  sentences  come  along  tvio 
action  can  be  taken:  establish  a new  class  or  move  an  initial 
substring  from  one  class  to  another. 

The  first  action  v;ill  occur  if  the  sentence  starts  with 
a u still  belonging  to  the  first  class  set  up  in  TABULA j but 
representing  a real  syntactic  variable  distinct  fro.m  the  initia 
symbol.  If  u has  not  yet  been  established  there  is  a positive 
probability  in  each  iteration  T that  it  will  be  discovered. 
Hence  with  probability  aero  this  v.'ill  happen  after  a finite 
number  of  iterations. 

The  second  action  means  that  it  is  discovered  that  two 
strings  u and  v,  that  are  put  in  the  same  class  temporarily, 
will  be  detected  to  be  non-equivalent  but  v is  beliovod  (for 
the  mo.T.cnt)  to  be  equivalent  to  some  w belonging  to  an  already 
established  clai's.  Then  v will  be  moved  to  the  class  of  _w. 

In  order  that  this  rdnould  happen  let  us  assu.me  that  all  classes 


have  been  created  already  vaiow  that  thiu  v;lll 
finite  time).  Duo  to  the  finite  depth  only  a firii 
arrancomonte  e:-;lot  for  the  c^’oepini:  of  the  initial 
For  each  one  there  io  a pooitive  probability  that 
croupinr.  cdrall  b.?  detected  in  one  trial.  ;Ionco,  a 
the  equivalence  claaoes  will  be  correctly  eotablio 

finite  nu.mber  of  iterations,  a^ain  with  probabilit 
This  concludes  the  proof  of  converf;ence  but 


happen  after 
to  nu.mber  of 
ctrinr's . 
an  incorrect 
s T -I-  all 
hod  after  a 
y one. 

it  should  be 


noticed  that  the  positive  probabilities  mentioned  may  be  small 
which  will  result  in  a very  low  learning  rate. 

It  is  clear  that  this  abduction  machine  has  only  a modest 
memory  requirement  compared  to  the  earlier  one.  Old  sentences 
need  not  be  remembered,  nor  is  it  necessary  to  store  the  enormous 
matrix  tha.t  represented  the  syntactic  relationships  i.n  qua.ntitive 
terms  of  strength  of  belief. 

The  learning  mechanism  also  appears  less  artificial.  It 
still  has  the  LISTEK-SPE.VK  cycle  but  the  spoken  sentences  are 
being  corrected  in  a less  systematic  m.anner. 

It  should  also  be  remarked  that  no  claim  is  made  that  the 
original  grammar  is  learnt  in  exactly  the  sa.me  for.m  as  it  has 
been  defined.  The  claim  is  that  as  it  converges  in  the  sense  of 
weakly  cquivalenv.  gvamjnars . 


As  a matter  of  fact  the  search  for  equivalence  classes  will 


result  in  a limiting  gi’hm.mar  with  a minimum  number  of  variables 


or  classes. 


In  this  sense  the  abduction  machine  appeals  to 


Occam's  rasor. 


To  tc3t  for  opeod  of  convergence  the  machine  wao  exposed 
to  various  finite  state  languages.  The  speed  of  learning  was 
much  higher  in  general  than  for  the  naive  .machine.  For  example, 
the  grammar  (f.l),  which  took  very  long  to  loam  before  was  now 
learnt  al.most  iiwicdiately . For  T=1  the  variables  D and  D3  were 
established,  for  T=2  the  variables  A,  for  o = 3 the  variable  3C : 


TADULA 
LIhGUA 
Hnlf  VARIA3LE 
HEl'/  VARIA3LE 
KEV;  VARIA3LE 
NEW  VARIABLE 


20 

3 CREATED  AT  SENTENCE  NO.  1 
33  CREATED  AT  SENTENCE  NO.  1 
A CREATED  AT  SENTENCE  NO.  2 
3C  CRi^ATED  AT  SENTENCE  NO.  6 


BDAnuAA33B 

33AB3AA333 

A333 

DC 


In  other  runs  the  perfor.r.ance  of  the  abduction  machine  was 
similar,  in  no  case  requiring  more  than  10  iterations  before 
convergence  was  established. 

For  other  finite  state  grammars  'with  many  variables  and 
rules  the  time  it  took  v;as  longer  correspondingly  but  no 
exorbitant  number  of  iterations  was  required  in  this  scries  of 
tests. 

I would  like  to  report  on  one  test,  not  because  the  gra-mjnar 
was  more  complicated^ but  because  the  result  seemed  surprising  at 
first  and  led  to  some  reflections  on  the  notion  of  style  that 
ought  to  be  explored  in  more  depth. 

I had  been  locking  at  languages  -where  the  sentence  is  not 
Just  expressed  as  a linear  string  of  words  but  takes  the  for.m  of 
a colored  picture.  ’Aore  precisely,  the  question  -was  whether  one 


r 
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could  find  syntactic  rules  that  v/ould  Generate  at  least  typical 
fracments  of  hir.nly  stylized  pictures,  cay  in  the  style  of 
Mondrian. 

The  tried  v;as  the  followlnc  one.  The  v/ords  wore 

five  A,D,C,D,  and  M with  the  rules  in  the  wirinG  diaGran  of 
Figure  3.3-  Generate  tv.-o  sentences  fro.T.  this  finite  state 
automaton,  code  A,3,C,D,E  into . 0 , 1 , 2 , 3 , ^ with  the  color  key 


3 - yellow 
^ - black 


Use  one  sentence  for  horizontal  effects,  the  other  one  vertically 
and  add  the  key  values  modulo  5-  Pictures  are  then  obtained 
looking  like  the  one  in  Figure  3-4.  It  can  be  questioned  whether 
this  is  really  Mondrian-like,  but  .this  is  not  the  point  hers. 

Vrnen  the  abductio.n  machine  was  exposed  to  sentences  con- 
vergence seemed  to  be  much  slower  than  for  other,  .see.mingly  more 
difficult  languages.  It  could  look  like  this 


I 


Flf-urc  3-  3 


O f; 
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I 


'SAPULA 


I.:  non  A 


7.0 


A DAD  ADA  A 
nVM  VAPIAPI.l' 
a I'M  vAUfAPr.i- 


A CPP/.TKD  AT  CKPTl'PCP  lin , 
AD  CPPATKP  /ir  'jr.UTF.ilCi:  :!0 


A V A "'A  PA  A 
- AHA  PAP  A A 


A FA  A 

.1  FA  DA  DA  />■/,  DA  FA  DA  CA  PA  PA  A 
ACA.FAFAFACACAA 

/'.  PA.  HA  PA  FA  FA  PA  PAPA  FA  FA  CA  PA  CA  FA  CA  FA  DA  DA  EA  FA  PA  FA  PA  FA  PA  PA  FA  PA  PA  CA  CA  CA  CAA 
A CA  CA  CA /I 

A FA  EA  EA  CA.  PA  CA  CA  EA  FA  CA  PA  FA  CA  PA  PA  FA  CA  D A CA.  CA  A 

A DA  CA  DA  PA  PA  PA  CA  FA  CA  CA  EA  A 

APA.iACACAPAA 

A CA  CA  EA  PA  PA  EA  DA  CA  PA.  CA  DA  PA  EA  CA  EA  PA  EA  FA  FA  PA  PA  A 
APAPA.Ih'M'AA 


A PAP  A DA.  A 

AFACADACAPAC/.PAA 
A CA  DA  CA  CA.  /)/!  ..'A  FA  CA  CA  A 

A FA  ,-7/l  DP.  CA  FA  FA  FA  FA  PA  CA  CA  DA  EA  DA  CA  DA  EA  EA  CA  CA  CA  P.A  /> 


A-.PACA.CA.CAA 
AFAPEADAFA.FAPAEAA 
AF AD AC A PA  FA  A 
A .'1 

p:::/  VAPrAP.LF  aa  cpa'.aptfd  at  e 

AFAEACAA 


:iCF  r.o 


: ■)  = 


v;il;h  only  A,y\D,AA  as  established  variables  after  20  iterations. 
Further  iterations  yielded  no  nore  variables.  The  sentences  tend 
to  be  rather  lor.f, , in  one  extreme  case  over  100  v;ords  long,  but 
this  should  only  marginally  influence  the  computing  time  needed 
since  most  of  it  goes  into  processing  short  initial  substrings. 

The  solution  is  quite  simple.  The  graimr.ar  recovered  is 
indeed  v/cakly  equivalent  to  the  given  one  as  can  be  seen  by 
inspection  of  the  wiring  diagram  in  the  automaton  of  Figure  3.5* 
The  abduction  machine  has  reconstructed  a correct  grammar  although 
in  a different,  simpler  form,  equivalent  but  not  equal  to  the  one 
v;e  started  out  from.  It  therefore  seems  that  we  could  just  as 


ii 


r igurc  3.5 


well  have  started  out  with  the  siniplcr  I'or'.r.  originally. 

Not  so.  If  v,’o  had  done  this  it  would  not  have  been 
possible  to  differentiate  the  probabilities  associated  with  the 
transitions  between  different  colors  and  bands  of  different  wid 
This  would  have  meant  that  we  could  not  have  incorporased  such 
.stylistically  r.i/'.nif leant  elements. 

This  lesson  teaches  us  that  the  net  ion,  of  style  that  v;f 
have  adontod  here,  Is  not  so  much  a nronortv  of  the  rra.TLma'r , 


as  the  usage  of  I'rammar.  It  is  rel 


ccmoetence 
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If 

case  for 
Increase 


this  point  of  view  is  accepted  it  rfioans,  as  was  the 
the  Mondrian  nachine,  that  it  rr.ay  bo  nocessary  to 
the  set  of;  syntactic  variables  by  o!:h-.--r  ••  1 7 ], i s tic 


variables . 


Tills  raises 
"efficient"  styli 
than  this  rather 
abduction  raachln'^ 


the  question  of  how  to  fiiui  adcciuate  a.nd 
Stic  variables  in  situations  of  {greater  i 
artificial  example.  It  is  clear  that  the 
learns  nrarnmar  but  not  style.  Can  one  f 


ntcros 


ormali 


ZQ 


the  search  for  stylistic  variables? 


It  stores  the  information  in  the  form  of  arrays  LEXICOM,  PfJLES, 


LEN,  CUiM.  In  ord<^r  to  save  space  the  user  has  been  restricted 
to  at  most  15  'words,  9 variables,  and  5 ro'.vritin^’;  rules  for  each 
variable.  If  this  is  not  enouch  the  statements  [1],  l9j,  [10], 
and  [12]  should  be  chanced  accordincly.  Ihc  initial  variable 


X O 


the  one  labelled  1. 
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i.fter  calling  SETUP  execution  of  the  function  REV.’RITE 
will  generate  one  grarunatical  string  naj7.Gd  ETRIMG  according  to 
the  model  STRING  ir,  a lexical  vector,  but  if  one  needs  a 
numerical  vector  this  is  easily  obtained  by  the  statement 

LEXICON  x STRING 

resulting  in  a vector  of  the  same  length  but  v;ith  each  word 
replaced  by  the  number  of  it  in  the  LEXICON, 


\j  n " -r  <Ti 

I 1 I ■’  ' r • N-  » ^ t 

r.:; 

f 3 I 

( H.l  ->(  /.?;,? 

!')'|  -)-o 

id;]  y;,;?  ■ 1 1 

r 3j  !'/i ir.  ’/vf  j/'h.'Vrn 

(■  ■>  1 ■rT//i ,?//,/? /;/;,•)  y,  :jy 

L ')  .1  ,7.7-,o- i-i-  + / r 'I'  •/.;  -7  r /.'. )/■:/  ; ] < ig-f,  x ? i o oo  0 o o 

r 1.  0.1  7, xLrUW  f/.'’ ■■■’ eg  ; g.vw]  j 

[111  lyjvTiux'?  /,7h[  i/,7’)gy-i]  ■:i!!G,.r'y::!n[.  rg-'g.r+l  g-  ri'n^v 

r 1 :!  I 
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Tlxe  function  ACCEPT  takes  an  input  string  SEN  as  an 
alphabet-lc  vector.  The  result  Z is  1 oi’  0 according  to  whether 


trlr.g  SEN 

is  j^rammatical  or 

ri,] 

1 PI 

; ' ' \ 

; >1 

Ai,y  yr’^T:([.  a?  i Li 

r.'hi 

rr’.l 

■■'.y  ’'■■-0)  /AL2 

[ i.  .1 

' vj 

"'•0 

rr.j 

•>■0 

1.  1 

AL?  :-(  0=p.'.-,7P)//1/,3 

f 1 0 1 

f'  1 1 ,1 

•M) 

(1 

A'  ! : 
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It  user,  an  array  MATRIX  whoso  rows  are  the  nunihers  (in  LEXICON) 
of  the  v;ortls  anh  whose  columns  arc  the  states  1,2,3,...  if 
necessary  increased  with  additional  states  as  described  in  the 
text.  The  entries  are  the  next  state  where  0 stands  for  the 
final  state.  The  automaton  in  Figure  2.1  for  example  'would  have 

H 
0 
0 


MATRIX  = 


The  function  TABULA  sets  uo  one  single  eeuivalence  class 


and  some  selected  variables.  The  matrix  CLA33 
number  of  columns  = n . 

r. 

fl.l  ( (:  n ) o . r:\.n)+ . x-,’:':'.- ir 

/7(:hA<-Ci/[h+i  j 
:i  ) AV^-i 

\ “ I CLAJO-^-d,  ilCOL)r<^ 

] '"ir.'U-i 


have  NCO: 


The  function  LINGUA  has  one  argument  MORE  = number  of 
additional  scntfuiccs  to  be  generated  and  presented  to  the  listener. 
It  uses  tlic  subroutj.nos  CODE  which  talccs  the  number  of  the 
substring  and  produces  the  substring  and  DECODE  which  is  the 
inverse  function,  both  for  substring  in  the  form  of  nu.merical 
vectors. 


r 1 ! 


/ :'.>cv 
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LINGUA  prints  out  each,  sentence  heard  and  calls  the  cor.parison 
function  TEGT. 


V LINGUA  MORE 

[1]  T-1 

[2]  LLli REWRITE 
[ 3 J STRING 

[ I*  J SFNT*-LEXI CO  N \ STRING 
[ 5 J TAKE->-l 

[6J  LL2:BEGIN^,SENTL\TAKE1 
C7j  V^-DECOOE  BEGIN 
[8j  SET*-CLASSiCLASSl  -.Vl/xLV-,  '] 
C9J  SET-^-SETil  -.l/xUCOL 
CIO]  y’Aury'*-5£:2’[  .’(p^FDC  i]J 

[11]  TEST 

[12]  TAKE-*-TAKE+l 

[13]  -y(TAKE^DlpSENT) /LL7 

[14]  T-f-r+i 

[15]  TAKE->-l 
LlGl  TO"'*-TOT+l 
[I7j  -yiIi.MORE)  /L:.  - 

7 


TEoT  prints  out  oa.ca  decision  to  introduce  a ncv.'  class,  the 
substring  corresponding  to  it,  the  sentence  nurr.bcr  v.'hen  it 
occurred,  and  the  sentence  itself. 

Afterwards  typing  CLASS  will  print  out  the  CLASS  array 
'Which  tells  us  ho'.-i  the  classes  that  have  boon  established  up 
till -now  are  co.nposed  from  substrings.  This  -will  enable  us  to 
easily  reconstruct  the  corresponding  rewriting  rules  and  the 
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wirlnG  diar.ram  of  the  finite  state  automaton  that  cenerates  the 
lancuacc  as  it  has  been  learnt  up  till  now. 


7 test 

[ 1]  U*-COnF.  TESTV 

[2J  YES*-ACCEPT  EEXI  CON\.U  .TAKE  iSENT] 

[3]  -^YES/O 

[ '4  J COMP->- , ( CL  AC,  El  •,Vi=0)/\LV 
[5J  TLl:-*(0=pCOMP) /TL?. 

[6  3 ROW*-{  CLASSl  COMPl  1 ] ; J ) / i ECOL 
L 7J  ROW->-lWWl  ?pRuy:i 
103  TESThf-^CODE  EOi^lll 

1.9]  -*(.~ACCEPT  LEXICONlTEETW.TAKE^SENTl) /TL3 
::  lOj  CLASSl  ( CLASSl  ; /]  = ! ) / i />/; 
i:il]  CLASSl  COMPl  1 ] iV]*-! 

'123  -►O 

■ 133  TL2:CLASSHCLASSl  Vi  = 1)  / x LV 
.14]  LV^LV-^1 

1.153  'NEW  VARIABLE  \ LEXT  COKlBEGI  N I V CREATED  AT  SESTERCE  SO.  '■,TOT-.'=  '.STRING 
i 163  CLASS*-(.LV,NCOL)p(.,CLASS),V^.xNCOL 

17]  -*0 

18]  TL3  : COM P->-l* COMP 

! 19]  -*'111 
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